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Figure 1: We transform a scanned input mesh into a stylized output emphasizing anatomy and characteristic features. The input is segmented 
into nearly planar regions via alignment with a template model The angular definition of the segmented mesh is enhanced in a highly 
controllable optimization which generates smooth full resolution output interactively. 


Abstract 

Sculptors often deviate from geometric accuracy in order to en¬ 
hance the appearance of their sculpture. These subtle stylizations 
may emphasize anatomy, draw the viewer’s focus to characteristic 
features of the subject, or symbolize textures that might not be ac¬ 
curately reproduced in a particular sculptural medium, while still 
retaining fidelity to the unique proportions of an individual. In this 
work we demonstrate an interactive system for enhancing face ge¬ 
ometry using a class of stylizations based on visual decomposition 
into abstract semantic regions, which we call sculptural abstrac¬ 
tion. We propose an interactive two-scale optimization framework 
for stylization based on sculptural abstraction, allowing real-time 
adjustment of both global and local parameters. We demonstrate 
this system’s effectiveness in enhancing physical 3D prints of scans 
from various sources. 

1 Introduction 

The technology required for a “3D fax” that can scan the shapes 
of household objects of various sizes, transmit them electronically, 
and reprint facsimiles of those objects at a distant location, is now 
available at a price feasible for many households. In addition to util¬ 
itarian uses such as printing missing or broken parts for household 
objects, 3D scanning and printing tools are also being employed 
for more personal items such as figurines of family and friends 1 . 
Although the market for such tools is nascent, we suggest that given 
the appropriate tools and low enough pricing in the future, we might 
be just as likely to send our relatives a 3D printed figure of a family 
vacation as we would a framed photograph: Just as photography 
democratized 2D imagery by unchaining it from the painter’s hand, 
3D scanning and printing may democratize 3D representations by 
decoupling them from the sculptor’s hand. 

Thus far, research on 3D scanning and printing has rightly focused 
on geometric accuracy. However, 3D printed human figures of- 


e.g., www.shapify.me,www.shapeways.com/tutorials/shapeme, 
www.fablitec.com 


ten appear lifeless, particularly when generated with commodity 
scanning and printing. We propose that creative applications of 
these technologies require further exploration of sculptural styl¬ 
ization , which we define to be when a 3D model deviates from 
geometric accuracy in a manner analogous in many ways to that 
of non-photorealistic rendering for 2D imagery. Sculptural styliza¬ 
tion could be either for purely aesthetic purposes; or to improve 
the recognizability of the subject by emphasizing characteristic fea¬ 
tures [Winnemoller et al. 2006]; or to fit the limitations or require¬ 
ments of a given sculpting medium; or simply to imitate historically 
popular styles of sculpture. Well known sculptors consistently ex¬ 
aggerate away from geometric accuracy in order to achieve some 
combination of these aims (a preliminary analysis of how sculpted 
busts deviate from real human geometry is provided in Section 3.1). 

As described in subsequent sections, one important class of styliza¬ 
tions first decomposes a model into masses, planes or other compo¬ 
nents. These decompositions could be purely geometric, but are 
often based on higher-level object semantics or a knowledge of 
underlying anatomy. We use the term sculptural abstractions to 
describe these semantically-based decompositions, using the word 
abstraction in the sense of isolating essential qualities. 

In this paper we propose an interactive approach to stylization of 
human faces, founded on a specific sculptural abstraction used by 
artists called the planes of the head — regions of the surface of the 
face which, although not literally planar, may be grouped together 
by geometric consistency or separated by underlying anatomy. We 
generalize this concept to scans of other objects beyond simply 
faces, and refer to the general concept as sculptor's planes. Our 
method’s stylizations emphasize the contrast between these forms 
by accentuating the angles between them and making the regions 
within them more planar. We imagine an expert sculptor who 
first studies his subject and identifies these sculptor’s planes, then 
produces a rough, low-detail model in which the surface angles 
between planes are exaggerated and deviations within planes are 
smoothed, and finally introduces fine details of the surface. 

Our system, using a 3D scan of the subject as its input, supports 



an amateur digital sculptor in several stages: First, the sculptor’s 
planes are automatically identified — either by aligning a scan to 
a generic, pre-segmented head model, or by automatic geometry 
analysis; and the scan is then simplified to an abstracted mesh us¬ 
ing the given segmentation. Second, we construct and optimize 
an energy function to balance the exaggeration of angles of this 
abstracted mesh with preservation of the original form and some 
key individual facial characteristics. The sculptor can adjust both 
the global scale and local scale of exaggeration as well as fidelity 
to the facial characteristics. Finally, we map the resulting deforma¬ 
tion back to the original full resolution scan, and the sculptor can 
locally adjust the the amount of detail to include from the original 
scan, as well as the smoothness between sculptors planes. The last 
two stages are performed continuously at interactive frame rates, 
providing detailed user control of both global and local parameters. 

Our contributions include an algorithmic analysis of stylization 
based on sculptural abstraction, and an interactive system to em¬ 
ulate these techniques under real-time user control of intuitive pa¬ 
rameters. We show that our approach permits a wide array of both 
subtle and exaggerated stylizations on both faces and other mod¬ 
els, and illustrate qualitative improvement over previous geometry 
deformations applied to faces. 

2 Related Work 

The non-photorealistic rendering literature features a long history 
of transforming photographs and 3D models into artistic depic¬ 
tions. In particular, Gao et al. [2013] used an abstracted model 
similar to the planes of the head for 2D stylization and exaggera¬ 
tion. Winnemoller et al. [2006] demonstrated that certain types of 
abstraction improve human memory tasks, including facial recog¬ 
nition and scene recall. We hypothesize that sculptural abstraction 
can play a similar role for 3D shape. Furthermore, many papers 
in NPR explore the depiction of 3D scenes to improve shape un¬ 
derstanding. Hertzmann and Zorin developed a hatching technique 
to emphasize aspects of curved surfaces that are otherwise hard to 
perceive [Hertzmann and Zorin 2000]. Cole et al. [2008] analyzed 
where people draw lines when representing 3D surfaces, which 
might be similar to the features that sculptors exaggerate. 

We are unaware of previous work that defines the broader notion 
of sculptural abstraction in 3D geometry. However, the idea of 
3D collage [Gal et al. 2007] seems to fall within this scope, as it 
attempts to represent the gestalt of a given 3D shape by aggregating 
several smaller 3D primitives. Bhat et al. [2004] also applied a 
method similar to image analogies [Hertzmann et al. 2001] in the 
context of 3D geometry. This method applied a learned geomet¬ 
ric texture to an entire model rather than local deformations based 
on existing geometry, and thus could be used as a complement to 
our sculptural abstraction in order to reproduce the surface texture 
imposed by various real-world sculpting tools. 

Other existing methods for automatic non-metric mesh enhance¬ 
ment take approaches which have predictable and global effects. 
Blanz and Vetter [1999] fitted a parametric 3D model built from a 
large family of faces to a new scan, and could create caricatures by 
shifting the parameters away from the mean face. This results in 
a global linear deformation, in contrast to our nonlinear template- 
driven deformation. PriMo [Botsch et al. 2006] is another defor¬ 
mation strategy that has been used to demonstrate caricature, by 
modeling polygons as prisms of finite thickness and constructing 
an energy function to increase angles between them. This resem¬ 
bles our exaggeration energy term, but although our system can be 
used for stylizations such as caricature — e.g. exaggeration of key 
individual features — we characterize our goal of abstraction as 
sharpening visual contrast and form without modifying key indi- 



Figure 2: Left: A scan acquired using KinectFusion. Right: The 
same scan after sculpting by Gio Nakpil. Note exaggerated features 
such as cheekbones, temples, jawline, chin, and sides of the nose. 

vidual features. This distinction is explored in detail in Section 4.2, 
in which we propose Lanteri constraints for maintaining fidelity to 
these features. 

Various signal processing techniques exist for meshes which can 
exaggerate features. The simplest is the 3D equivalent of the 
unsharp mask, where a 3D shape is enhanced by smoothing its 
normals and then exaggerating their angles in the opposite direc¬ 
tion [Yagou et al. 2003]. Eigensatz et al. [2008] operate in the 
curvature domain, but do not process meshes in real time. More 
recently, anisotropic smoothing and exaggeration have been imple¬ 
mented in graphics hardware [Chuang and Kazhdan 2011]. Al¬ 
though these methods can produce some results similar to ours, they 
don’t offer simple local control. 

Some previous works have explored automated abstraction of 
shapes [Mehra et al. 2009] and shape collections [Yumer and Kara 
2012] into semantic groupings, though they do not demonstrate 
stylizations based on these groupings. Similarly, Lee et al. [2000] 
decomposed geometry into a base mesh plus displacements not un¬ 
like our two-scale decomposition, but their model is not suitable for 
deformation because it associates each high-resolution vertex with 
only one base polygon. 

3 Background 

In order to learn what techniques sculptors use in their abstrac¬ 
tion, we interviewed two accomplished professionals: Gio Nakpil, a 
sculptor who works both in physical and digital sculpture, including 
work for Industrial Light and Magic and Valve; and Mike Magrath, 
an instructor at Seattle’s Gage Academy of Art. Although many 
techniques are particular to specific styles, we narrowed our focus 
to abstraction techniques for human busts that span multiple styles. 

Two common points of reference between the sculptors emerged: 
First, both sculptors mentally and visually decompose a subject 
into component masses or forms at a variety of scales — starting 
from larger groupings such as the forehead, and working down to 
smaller details like the sides of the nose and the folds of the eyelid. 
And second, in the case of human faces this decomposition is often 
described in terms of planes of the head , as defined in Section 1. 

Nakpil emphasized the role of highlight and shadow in sculpting 
technique 2 , noting that sculptors view their work under a range of 
lighting conditions and from many angles to understand and control 
the shapes and relationships of those highlights and shadows. Since 

2 In this context, highlight and shadow refer to the broad artistic definition 
of regions brighter or darker than the mean, not only specular highlights 
and cast shadows. 
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Figure 3: Left: Examples of planar decompositions in painting and 
sculpture, ©John Asaro 1976, excerpted with permission. Right: A 
variation of the Planes of the Head entitled Memorized, intended 
for student memorization. 


sculptures are typically lit from above, sculptors may tilt surfaces 
toward the horizontal, exaggerating the contrast between highlights 
and shadows. Figure 2 shows a scan we provided to Nakpil along¬ 
side his sculptural interpretation using ZBrush. 

Magrath made special note of the exaggeration of “soft” and “hard” 
forms. This nomenclature refers to the distinction between angu¬ 
lar regions with rapidly alternating high and low curvature ( e.g ., 
the bridge of a nose) versus rounded, uniform medium-curvature 
regions (e.g., the forehead). He characterized his abstraction tech¬ 
nique as “making the hard forms harder, and the soft forms softer.” 

The notion of planes of the head permeates literature about sculpt¬ 
ing, and dates back at least to the turn of the 19th century, when the 
sculptor Edouard Lanteri (whom Rodin described as “Dear Mas¬ 
ter” [Lanteri 1911]) wrote a seminal textbook. Although much of 
his work describes technical methods for ensuring metric accuracy, 
he also suggests a methodical approach to exaggerating these planes 
and their junctions using side lighting to reveal them more clearly 
to the sculptor: 

When these divisions of form have been obtained in their 
proper drawing by studying each separately, the work may 
appear a little hard. Then it becomes necessary to work by 
colour — that is to say, by the comparative values of the half¬ 
tints, in simplifying or accentuating the surfaces or planes 
which divide these forms. [Lanteri 1985] 

The concept of planes of the head was further formalized in the late 
20th century, most notably by John Asaro, an instructor at Art Cen¬ 
ter College in Pasadena, California [Asaro 1976]. Asaro hypoth¬ 
esized that decompositions of facial form into masses and planes 
have been used by painters and sculptors since antiquity, and pro¬ 
posed planar decompositions for works by Cassatt, Rodin, Degas, 
Rubens, Vermeer, Michelangelo and many others. As an aid to stu¬ 
dents of sculpting and painting, Asaro developed a set of canonical 
busts which exaggerate the planes to their logical extreme, reducing 
a human head to a nearly polygonal object (see Figure 3). 

While sculptors use these planes to exaggerate certain forms, 
Lanteri’s writing also makes it clear that certain aspects of human 
anatomy and individual facial characteristics are perceptually rel¬ 
evant, and must not be perturbed by these stylizations. He writes 
voluminously about using calipers to ensure that certain distances, 
ratios, and angles are preserved in a sculpture of a given model. 
Here is one such passage: 


From the tip of the nose measure the distance to the most 
projecting part of the chin, i.e. the subcutaneous mental 
eminence... To make sure of the correctness of the previ¬ 
ous measure, you now take the distance from the ear to the 
mental eminence (see Fig. 40) in your calipers, and if this 
measure, taken from both ears of course, coincides with the 
previously fixed point on the chin, your measure runs a fair 
chance of being correct. If it is not, you have to... retake 
both measures, until they agree. [Lanteri 1985] 

Additional measurements proposed by Lanteri appear in Appendix 
A. We make use of these in our algorithm to preserve the facial 
characteristics of the subjects during stylization. 

We hypothesize that the planes of the head perform a similar func¬ 
tion to lines in 2D illustration: Artists abstract detail that isn’t im¬ 
portant and emphasize features that do matter for recognition and 
understandability. Although we may not yet know exactly what 
features are perceived as important for emphasis in 3D, we choose 
to follow methods employed by professional sculptors because it is 
reasonable to assume they relate to perceptual importance. 

Thus the challenge of sculptural abstraction is to enhance or exag¬ 
gerate visual contrast between sculptor’s planes, while simultane¬ 
ously preserving certain critical global geometric relationships. Al¬ 
though it may seem these goals are fundamentally incompatible, in 
the sections that follow we demonstrate a system that accomplishes 
this automatically, interactively and under flexible user control. 

3.1 Analysis of scans of humans vs. sculptures 

We observed informally that, in spite of some sculptors’ focus on 
geometric fidelity, some features in realistic classical and modern 
sculptures appear exaggerated. This is hard to detect when compar¬ 
ing people directly to sculptures, because their reflectance qualities 
differ so drastically. However, these differences become more ap¬ 
parent when people and sculptures are scanned. Perhaps most crit¬ 
ically for our application, 3D prints of scanned faces often appear 
flat and lifeless. 

An ideal analysis of sculptural stylization would require extensive 
data collection of existing sculptures and comparisons to human 
scans of the individual models being sculpted. In the absence 
of such a dataset, we performed an informal comparison of the 
depth of the eye sockets between six scans of humans - four from 
KinectFusion [Newcombe et al. 2011] and two using the method 
of Beeler et al. [2010] - and five scans of sculptures - three of 
Rodin’s “Burghers of Calais” 3 , and two busts from the MIT CSAIL 
3D Model Database 4 . First, a number of feature points on the mod¬ 
els were identified and manually labeled. Then, we measured the 
depth of the eye sockets using four separate scale-invariant mea¬ 
sures: Measure A takes the distance from the midpoint of the inner 
corners of the eye to the plane formed by the midpoint of each eye¬ 
brow and the tip of the chin, normalized using the distance between 
the points at the base of the ears. Measure B is the same as measure 
A, but using the outer corners of the eyes. Measure C takes the 
distance from the midpoint of the inner corners of the eye to the 
plane formed by the saddle point at the bridge of the nose and the 
corners of the mouth, also normalized using the distance between 
the ears. Measure D is the same as C, but using the outer corners of 
the eyes. Each measure was aggregated across human and sculpted 
scans using both the mean and the median. A summary of the re¬ 
sults can be found in Table 1. 


3 http://www.stanford.edu/~qianyizh/projects/scenedata. 
html 

4 http://people.csail.mit.edu/tmertens/textransfer/data/ 
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A 

B 

c 

D 

Mean 

Human 

0.069 

0.134 

0.094 

0.157 

Sculpt 

0.112 

0.177 

0.131 

0.193 

% Increase 

63.1 

32.5 

39.3 

23.1 

Median 

Human 

0.065 

0.127 

0.091 

0.156 

Sculpt 

0.091 

0.169 

0.127 

0.183 

% Increase 

40.2 

33.5 

40.4 

17.4 


Table 1: Measures of the depth of the eye sockets on scans of 
humans vs. sculptures. 


For each aggregate measure, the sculpted scans had significantly 
deeper eye sockets than the human scans. Note that all fiducial 
points were identified by hand, the quality and resolution of the 
scans varies significantly, and the measures we propose are some¬ 
what ad hoc. Furthermore, the size of the sample is much too small 
to draw concrete conclusions. Nonetheless we see a strong sugges¬ 
tion that sculptors such as Rodin routinely exaggerated features of 
facial anatomy such as the depth of eye sockets. 

4 Approach 

Our goal is to allow a user to interactively stylize an input face 
model guided by sculptural abstraction principles. Our system first 
computes an abstracted mesh from the input face model. The ab¬ 
stracted mesh needs to convey both high-level abstract elements of 
the input mesh and semantically meaningful facial features. To this 
end, we use a Planes of the Head model as our template to guide 
our abstracted mesh generation. The abstracted mesh serves as 
the sculptural abstraction framework for the subsequent stylization 
steps, and is divided into meaningful regions corresponding to the 
planes from the template model. 

The subsequent sculptural stylization optimization and stylization 
transfer phases of our system run at real-time rates and are designed 
for interactive use by a user in control of global and/or local pa¬ 
rameter settings. We stylize the abstracted mesh by minimizing an 
energy function with several terms of various purposes: to exagger¬ 
ate angles between different regions; to enforce flatness within each 
segment; to regularize the stylization towards the original shape; 
and to enforce Lanteri constraints that preserve geometric measure¬ 
ments characteristic of a person’s identity. The transfer of these 
stylizations back to the input mesh produces real-time full resolu¬ 
tion results for inspection and further adjustment. The amount of 
smoothing can be controlled here both globally and locally; and the 
interactive display of the full resolution deformed mesh is crucial 
in permitting fine user control over subtle deformations. 

4.1 Abstracted Mesh Generation 

We first generate the abstracted mesh M c that captures locally 
meaningful sculptural abstractions from the input mesh M. For 
general 3D models, we find that Variational Shape Approximation 
(VSA) [Cohen-Steiner et al. 2004] often derives plausible geomet¬ 
ric abstractions from the input mesh by grouping faces of similar 
normals into contiguous regions. However for human faces, due to 
our heightened perceptions, VS A can cause noticeable artifacts be¬ 
cause it does not preserve semantically meaningful segmentations 
and salient features such as eyes and lips. 

Thus, we employ a semantically segmented human head template 
to generate the abstracted meshes from human face models. More 
specifically, we use the Planes of the Head model (Figure 3) be¬ 
cause it takes into account both the principles of sculptural abstrac¬ 
tion and facial semantics. To acquire the 3D template, we scanned 
a Planes of the Head model using KinectFusion [Newcombe et al. 
2011] and manually segmented the model into sculptor’s planes 



Figure 4: Planes of the Head: Memorized scan and segmentation. 



(a) (b) (c) 


Figure 5: The segmentation of the template model is transferred to 
the input mesh using nonrigid alignment. 


(shown with colored segments in Figure 4). For each novel input 
mesh M we transfer the segmentation by non-rigidly aligning M to 
the template, using the method of Li et al. [2009], and taking the 
nearest vertices as correspondences (Figure 5b). 

The transferred segmentation divides M into K regions {R r }f =l 
(K = 32). We generate the abstracted mesh M c by approximating 
each sculptor’s plane with a small number of triangles. We fol¬ 
low the approach of Cohen-Steiner et al. [2004]: anchor vertices 
are placed along the borders between sculptor’s planes and a con¬ 
strained Delaunay triangulation is used to fill the regions. Note that 
our algorithm only uses the anchor vertices on the boundaries be¬ 
tween regions, the triangulation is used for visualization purposes. 


4.2 Abstracted Mesh Stylization 


After we compute the abstracted mesh M c for the input mesh fM, 
we build up a stylization framework on the abstracted mesh in order 
to eventually transfer the stylization to the input mesh (Sec. 4.3). 
The stylization framework includes both exaggeration of the angles 
between the regions and planarization of the features within each 
region. Here we introduce a few definitions to facilitate our formu¬ 
lation of the stylization framework. 


We first define the sculptor’s plane n r approximating each region R r 
by the normal n r and centroid c r . c r is computed as the weighted 
average of the centroids of all the triangles within R r and n r as 
the average of the triangle normals weighted by the triangle areas 
within R r : 


Lt€R r A,n, 
E t£R r A,n, 


( 1 ) 


Since A t n t = \ ti x \ t2 + \ t i x \ t2 + \ t ?> x v*i where v*i, \ t2 , v f3 are 
the vertices of triangle t, all the terms with respect to the internal 
edges in Eqn. 1 cancel out given that V; x v ; = — v ; x x t . Thus we 
only need to include the terms with respect to each boundary edge 

e = Ki,v e2 ) e dRi: 


| = I eedRr y _e\ X Ve2 

|E«E3R r Vel X V e2 


( 2 ) 
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To facilitate the stylization transfer in (Sec. 4.3), we define an affine 
transformation T r for each R r as follows: 

T r = TX, (3) 

where T r r defines the rigid transformation implied by the change of 
n r to n' r and c r to c' in the stylization optimization in this section: 

T r = [R(n r , n') | c' - R(n r ,n')c r ] , (4) 

where R(n r , n') is the rotation matrix to transform n r to n' r . This 
can be derived from Rodrigues’ formula to compute the rotation 
matrix based on the axis n r x n' r and angle cos -1 (n r • n'). 

The planarization transformation T£ scales the affine space in the 
n r direction relative to c r : 

T?(#t) = [I 3 - nn r nj | fi(n r • c r )n r ] , (5) 

where jU is a user-defined variable which determines the amount of 
planarization and I 3 is the 3x3 identity matrix. If a point p is at a 
distance d from the plane then Tf (/x)p will be a distance (1 — ti)d 
from the plane n r . 

Note that we can derive all the quantities mentioned above from 
vertex positions. Our optimization in the following will be formu¬ 
lated in terms of vertex positions directly or indirectly. 

Energy formulation 

We pose the stylization of our abstracted mesh as an energy min¬ 
imization problem. We consider two types of energies in our for¬ 
mulation. The stylization energy exaggerates the angles between 
adjacent regions while keeping the regions themselves roughly pla¬ 
nar; and the regularization energy keeps the solution close to the 
original abstracted mesh and the Lanteri constraints as suggested 
by Lanteri [1985]. The total energy to be minimized is the sum of 
all stylization and regularization energies: 

E = Estyle ~\~ E reg . (6) 


Stylization energy 

The stylization energy consists of two terms for exaggeration be¬ 
tween adjacent regions Rj and Rj and planarization within each 
region: 

Estyle = Kl y WijUi ■ 11, + Xf y y (n r • (c r - v)) 2 . (7) 

Ri~Rj r \eR r 

Here, Rj ~ Rj means that region R f shares a boundary with region 
Rj. Weights wy scale the amount of exaggeration between Rj and 
Rj. X c \ (d for dihedral) controls the overall amount of exaggeration 
while Xf controls how flat the regions should be. 

The weights are set as the boundary length between R, and 
Rj normalized by the average boundary length. We find that this 
weighting scheme ensures that each edge’s deformation is weighted 
approximately according to its visual impact on the final model. 
Our system allows the user to alter these weights by specifying a 
scale factor s t j such that: wy sy-wy. 

We can transform the dot product between unit normal vectors 
n j • n j into an energy with a sum of squares form to facilitate the 
use of efficient solvers such as Levenberg-Marquardt: 

n i 4 n j = 1 ll n i + nj11 2 — 1 • (8) 


The edge and area terms are as follows: 

<m> 

E e dge = K ^ (l ~ Orl (11) 

eeMc ' \ e \ / 

where A(R r ) is the area of R r . \e\ is the length of edge e and the sum 
only includes edges on the borders between regions. The super¬ 
script “0” denotes the initial value. By using ratios with the initial 
value we make the terms scale invariant. 


The vertex and normal terms are simple L 2 error metrics: 

Evertex = Z ll v “ v 011 2 

^\ e \ vev c 


( 12 ) 


"normal — 


X n II 

= tLK 


(13) 


where \e\ is the mean edge length of the mesh used to make the 
vertex term scale invariant. 


All of these regularization terms are needed to avoid various degen¬ 
eracies of the energy function and maintain closeness to the original 
mesh, but the results are not critically affected by their weights. We 
used constant values of X v , X n , X e , and X a (given in Section 5) for all 
results shown in this paper. 

Lanteri constraints 


Based on the writings of Edouard Lanteri, we posit that certain rela¬ 
tive measurements on the face are critical to maintain the individual 
“personality” of the model (Figure 6). We thus consider three types 
of Lanteri constraints to formulate the Lanteri energy E Lanteri . An 
absolute position constraint (Eq. 14a) ensures that certain points of 
interest remain at their original positions, a relative position con¬ 
straint (Eq. 14b) ensures that the relative position of two points 
remains the same, and a relative distance constraint (Eq. 14c) en¬ 
sures a constant distance between two points during stylization. The 
energy is formulated as a sum of term of the following form: 


E 


K 

w 


II T (P)P-P 


E 

Pl,P 2 


K ll(T(pi)pi - T(p 2 )p 2 ) - (Pi - P2)|| 2 
2 ||Pi — P2II 2 


E 

P1P2 


K (IIT(pi)pi - T(p 2 )p 2 || - ||pi - P2II) 2 
2 ||Pi — P2II 2 


(14a) 

(14b) 

(14c) 


T(p) transforms p based on its closest regions’ affine transforma¬ 
tion (Sec. 4.3). The specific feature points used to formulate these 
constraints are given in Appendix A. 


Energy minimization 

To find the minimum of Equation 6 the energy is written as a func¬ 
tion of the vertex positions and minimized using the Levenberg- 
Marquardt method in Ceres Solver [Agarwal et al. ]. The vertices 
on the open boundary of the mesh are kept constant during the op¬ 
timization. Standard thresholds on function tolerance and gradient 
tolerance are used to determine convergence. 


Regularization energy 

The regularization energy E reg includes the following terms: 

1 4“ E ec j ge + E vertex T E norma i E Lanteri . (9) 


Ereg E area 


4.3 Stylization Transfer 

To transfer stylization of the abstracted mesh back onto the input 
mesh, we need to ensure smooth transitions between different re- 
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Figure 6: Lanteri constraints help to retain the personality of the 
original mesh (middle) in the stylized mesh (left). Without the con¬ 
straints unwanted deformations can occur such as the eyes getting 
closer together (right). 



Figure 7: Varying levels of stylization smoothness can enhance or 
smooth the boundaries between sculptors ’ planes while the orien¬ 
tation of the planes remains the same. 


gions to avoid any visible artefacts. We define a set of skinning 
weights Wi yT for each vertex v* with respect to each region R r . The 
final affine transformation T(v,-) for each vertex V/ of the input mesh 
is defined as the weighted average of all relevant T r : 

K 

T(v,-) = £ w,,T r , ( 15 ) 

r= 0 

where we define Tq to be the identity transformation which is 
assigned to the region outside of the optimization area to enable 
smoothing over this boundary. 



Figure 8: A user can adjust intuitive exaggeration and smoothing 
terms for specific edges (left) or planarization terms for specific 
faces (right) of the abstracted mesh. The resulting deformations 
are shown interactively on the full resolution mesh. 



Figure 9: Our system allows for local as well as global controls. 
Here , we show increasing exaggeration of the brow. 


To accomplish this at interactive rates, we adapt a method for con¬ 
trolling spatially varying blur from Diffusion Curves [Orzan et al. 
2008]: The user selects a boundary {/, j} between regions Rj and 
Rj and assigns a smoothing scale 5. All vertices on the boundary 
{/, j} are then assigned the value s. The system propagates the 
smoothing scale within both regions by solving a Laplace equation 
with Dirichlet boundary condition within each region: 

V 2 i = 0 (17) 

where the Laplacian operator is discretized by the cotangent Lapla- 
cian weights. This system is solved using a Cholesky decomposi¬ 
tion, and the solution is used to look up the smoothed weights from 
the skinning weight pyramid. Since the regions are typically small 
(two regions), and the Cholesky decomposition is independent of 
the value of s, the updates are very fast. 

4.4 Interaction 


Our system allows the user to adjust the stylization smoothness 
across different regions. Figure 7 shows one example where the 
user creates different stylization smoothness using our interactive 
user interface. To efficiently compute the skinning weights for in¬ 
teractive stylization smoothness editing, we pre-compute a pyramid 
of skinning weights w\ r with different levels of smoothing. Given 
a smoothing scale 5, we can then linearly interpolate the proper w^ r 
between the two closest levels of skinning weights w\ r and w-J 1 . 

The first level of the pyramid contains the original, un-smoothed 
skinning weights. In this case we set the weights w^ r to be the 
number of faces in R r that contain vertex v,-, normalized so that 
Hr w i,r = 1- For all vertices not on a region border there will only 
be a single non-zero weight. To compute the skinning weights at 
the next level, we perform simple Laplacian smoothing based on 
the adjacent skinning weights: 






(16) 


where V; ~ Xj means that V; is adjacent to Xj. The smoothing in¬ 
creases as a geometric sequence in the number of smoothing itera¬ 
tions as we go down the pyramid. 


After the pyramid has been precomputed, different amounts of 
smoothing can be interactively set for different areas of the mesh. 


Once we have aligned a new scan with the template model and 
produced its abstracted mesh, our full optimization framework 
for abstracted mesh stylization (Sec. 4.2) and stylization transfer 
(Sec. 4.3) runs at an interactive frame-rate of 10Hz on a standard 
desktop workstation. Our GUI (Fig. 8) provides users with intu¬ 
itive controls for the amount of exaggeration, the amount of pla¬ 
narization and transform smoothness. Each of these can be adjusted 
globally, or for more detailed control, exaggeration and smoothness 
amounts can be adjusted for each edge (Fig. 9) and planarization 
amounts can be adjusted for each face. These correspond to intu¬ 
itive visual changes, modifying the visual contrast between regions, 
displayed live in the form of the full resolution deformation result. 
Please see the supplementary video for a screen-captured demon¬ 
stration. 

5 Results 

The input triangular mesh can come from any source. The only re¬ 
quirement is that the areas to be abstracted are connected, manifold 
and contain no holes. We tested our system on meshes from two 
different sources; scans we made of a number of volunteers using 
the method of KinectFusion [Newcombe et al. 2011], and meshes 
obtained from a state-of-the-art multi-view stereo reconstruction 
system [Beeler et al. 2010], 
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Kd — 0.5 Xd — 1.0 Xd — 1.6 Xd = 2.3 

Figure 10: Increasing amounts of stylization, from left to right. We 
intentionally push the exaggeration too far to demonstrate how the 
Lanteri constraints keep salient features in correct positions. 



Figure 11: Increasing planarization (left to right) 



Figure 12: 3D printed busts — original (left) vs. stylized (right). 


The deformations produced by our method typically enhance 
cheekbones, deepen eye sockets, and emphasize the brows, chin 
and lips (see Figure 15). This approach is especially effective for 
the KinectFusion scans, which are sometimes lifeless and hard to 
recognize due to their lower fidelity. Further, materials used in 3D 
printing can be more translucent than skin, giving a blank look to 
printed busts. Our method gives such busts stronger definition by 
accentuating facial features, often making them more recognizable 
(see Figure 12). Our method can also be applied to models other 
than faces by using VS A for abstraction. In this case, the models 
are often thinned, and concavities become more pronounced (see 
Figure 13). The deformations shown in these figures are subtle, 
by design, and may be hard to see clearly in the printed paper. 
We direct the reader to view the electronic version, and especially 
to our supplementary video, which demonstrates the abstractions 
more clearly by cutting rapidly between inputs and outputs. 

For the stylization of the abstracted mesh (Section 4.2), the opti¬ 
mization is performed over a small number (n < 100) of vertices on 
the coarse mesh, and thus takes only a fraction of a second, regard¬ 
less of the input scan resolution. The stylization transfer step (Sec¬ 
tion 4.3) works with the full resolution mesh, but since only local 
linear solves and linear operations are required, both steps together 
can be run interactively. For an input mesh with 30k vertices our 
system runs at 10Hz on a Linux laptop, including all rendering, with 
room for further optimisation. For one of the Beeler et al .meshes 
containing 375k vertices the system runs at about 2Hz. Currently 
the system runs entirely on the CPU, however, the stylization trans¬ 
fer step would be well suited to GPU acceleration. 



Figure 13: Original mesh (left), segmentation (middle), and styl¬ 
ized mesh (right). Note the emphasis of facial features on the 
bunny 5 and sharper definition of the eye sockets on the lion 6 . 



Figure 14: A style can be transferred to other meshes by using the 
same set of weights. 


X v = 60, Xn = 1, and Xf = 1. The default exaggeration weights 
yield pleasing results with the user just specifying 0 < kd < 3 
to control the overall amount of deformation. However, the user 
can customize the output by specifying per-edge exaggeration and 
smoothing weights and per-face planarization weights. 

A video and a selection of the results (including original, seg¬ 
mented, abstacted and stylized meshes) are made available on our 
project webpage: http://wp.doc.ic.ac.uk/robotvision/ 
project/face-stylization/ 

6 Discussion and Conclusion 

Experts in sculpture will note that both the approach and algorithm 
we have chosen are both oversimplifications of a sculpting process. 
Our interpretation of the planes of the head, inspired by Lanteri and 
Asaro, is only one abstraction used by sculptors, and our algorithm 
is only one possible implementation of this abstraction. We hope 
and expect that future researchers will explore alternate interpreta¬ 
tions and methods with us. We expect that much further insight will 


Most of the weight parameters described in Sections 4.2 and 4.3 are 
held constant for all examples in this paper: X a = 10 and X e = 4, 
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5 Source: Stanford University Computer Graphics Laboratory 

6 Copyright purita, licensed under CC BY-NC 3.0. Source: http: //www. 
thingiverse.com/thing:215420 

















Figure 15: (a)-(d) KinectFusion scans; (e),(f) scans from Beeler et al. Top row: input; bottom row: output. 



(a) (b) (c) (d) (e) (f) 

Figure 16: (a) original mesh, (b)-(d) results from our system, (e) result from PriMo [Botsch et al. 2006], (f) result using the screened Poisson 
equation [Chuang and Kazhdan 2011 ]. Our system is capable of a wider range of custom stylizations: (b) bulbous nose, sharpened chin and 
accentuated brow, (c) planarization with low exaggeration, (d) rounder face, larger cheeks. 


be gained by extending the initial comparative study of the geomet¬ 
rical properties of real versus sculpted faces which we present in 
our supplementary material. 

Both Nakpil and Magrath described working at multiple scales in 
their sculpting work, from large planes to smaller forms. In this 
paper we have utilized only two scales, but combining deformations 
at multiple scales may produce a wide variety of sculptural styles. 

One significant limitation of our approach is that it works best on 
surfaces with a mixture of convex and concave regions. (As a trivial 
example, it is impossible to uniformly decrease all the angles of a 
convex polygon, because the sum of the internal angles is constant.) 
Further, our method is largely ad hoc , based on conversations with 
sculptors and our analytical interpretation of their intent. But we 
believe that given an appropriate dataset, it may be possible to learn 
sculptural abstraction in a more principled way from real data. 

This paper describes just one small step in the direction of sculp¬ 
tural abstraction. We look forward to further exploring the ways in 
which sculptors depict natural shapes, and learning effective geo¬ 
metric modifications that emulate these techniques. 
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Appendix A - Lanteri Constraints 

In selecting geometric constraints to preserve individual facial char¬ 
acteristics, we identified seven passages of Lanteri [1911, 1985] 
that correspond to nine relative measurements between points on 
the surface of the model. All of these passages refer to maintaining 
distances between points, but our optimization only modifies the 
face region, so measurements to points such as the ears or the top 
of the sternum are treated as absolute positional constraints rather 
than relative ones. Table 2 provides these seven passages, along 
with our interpretation of each as a distance constraint. 
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